Global rates of convergence for nonconvex optimization on manifolds

We consider the minimization of a cost function $f$ on a manifold $M$ using Riemannian gradient descent and Riemannian trust regions (RTR). We focus on satisfying necessary optimality conditions within a tolerance $\varepsilon$. Specifically, we show that, under Lipschitz-type assumptions on the pullbacks of $f$ to the tangent spaces of $M$, both of these algorithms produce points with Riemannian gradient smaller than $\varepsilon$ in $O(1/\varepsilon^2)$ iterations. Furthermore, RTR returns a point where also the Riemannian Hessian's least eigenvalue is larger than $-\varepsilon$ in $O(1/\varepsilon^3)$ iterations. There are no assumptions on initialization. The rates match their (sharp) unconstrained counterparts as a function of the accuracy $\varepsilon$ (up to constants) and hence are sharp in that sense. These are the first deterministic results for global rates of convergence to approximate first- and second-order Karush-Kuhn-Tucker points on manifolds. They apply in particular for optimization constrained to compact submanifolds of $\mathbb{R}^n$, under simpler assumptions.Comment: 33 pages, IMA Journal of Numerical Analysis, 201

Cram\'er-Rao bounds for synchronization of rotations

Synchronization of rotations is the problem of estimating a set of rotations R_i in SO(n), i = 1, ..., N, based on noisy measurements of relative rotations R_i R_j^T. This fundamental problem has found many recent applications, most importantly in structural biology. We provide a framework to study synchronization as estimation on Riemannian manifolds for arbitrary n under a large family of noise models. The noise models we address encompass zero-mean isotropic noise, and we develop tools for Gaussian-like as well as heavy-tail types of noise in particular. As a main contribution, we derive the Cram\'er-Rao bounds of synchronization, that is, lower-bounds on the variance of unbiased estimators. We find that these bounds are structured by the pseudoinverse of the measurement graph Laplacian, where edge weights are proportional to measurement quality. We leverage this to provide interpretation in terms of random walks and visualization tools for these bounds in both the anchored and anchor-free scenarios. Similar bounds previously established were limited to rotations in the plane and Gaussian-like noise

A Riemannian rank-adaptive method for low-rank matrix completion

The low-rank matrix completion problem can be solved by Riemannian optimization on a fixed-rank manifold. However, a drawback of the known approaches is that the rank parameter has to be fixed a priori. In this paper, we consider the optimization problem on the set of bounded-rank matrices. We propose a Riemannian rank-adaptive method, which consists of fixed-rank optimization, rank increase step and rank reduction step. We explore its performance applied to the low-rank matrix completion problem. Numerical experiments on synthetic and real-world datasets illustrate that the proposed rank-adaptive method compares favorably with state-of-the-art algorithms. In addition, it shows that one can incorporate each aspect of this rank-adaptive framework separately into existing algorithms for the purpose of improving performance.Comment: 22 pages, 12 figures, 1 tabl

Two Algorithms for Orthogonal Nonnegative Matrix Factorization with Application to Clustering

Approximate matrix factorization techniques with both nonnegativity and orthogonality constraints, referred to as orthogonal nonnegative matrix factorization (ONMF), have been recently introduced and shown to work remarkably well for clustering tasks such as document classification. In this paper, we introduce two new methods to solve ONMF. First, we show athematical equivalence between ONMF and a weighted variant of spherical k-means, from which we derive our first method, a simple EM-like algorithm. This also allows us to determine when ONMF should be preferred to k-means and spherical k-means. Our second method is based on an augmented Lagrangian approach. Standard ONMF algorithms typically enforce nonnegativity for their iterates while trying to achieve orthogonality at the limit (e.g., using a proper penalization term or a suitably chosen search direction). Our method works the opposite way: orthogonality is strictly imposed at each step while nonnegativity is asymptotically obtained, using a quadratic penalty. Finally, we show that the two proposed approaches compare favorably with standard ONMF algorithms on synthetic, text and image data sets.Comment: 17 pages, 8 figures. New numerical experiments (document and synthetic data sets

Searching for faint companions with VLTI/PIONIER. I. Method and first results

Context. A new four-telescope interferometric instrument called PIONIER has recently been installed at VLTI. It provides improved imaging capabilities together with high precision. Aims. We search for low-mass companions around a few bright stars using different strategies, and determine the dynamic range currently reachable with PIONIER. Methods. Our method is based on the closure phase, which is the most robust interferometric quantity when searching for faint companions. We computed the chi^2 goodness of fit for a series of binary star models at different positions and with various flux ratios. The resulting chi^2 cube was used to identify the best-fit binary model and evaluate its significance, or to determine upper limits on the companion flux in case of non detections. Results. No companion is found around Fomalhaut, tau Cet and Regulus. The median upper limits at 3 sigma on the companion flux ratio are respectively of 2.3e-3 (in 4 h), 3.5e-3 (in 3 h) and 5.4e-3 (in 1.5 h) on the search region extending from 5 to 100 mas. Our observations confirm that the previously detected near-infrared excess emissions around Fomalhaut and tau Cet are not related to a low-mass companion, and instead come from an extended source such as an exozodiacal disk. In the case of del Aqr, in 30 min of observation, we obtain the first direct detection of a previously known companion, at an angular distance of about 40 mas and with a flux ratio of 2.05e-2 \pm 0.16e-2. Due to the limited u,v plane coverage, its position can, however, not be unambiguously determined. Conclusions. After only a few months of operation, PIONIER has already achieved one of the best dynamic ranges world-wide for multi-aperture interferometers. A dynamic range up to about 1:500 is demonstrated, but significant improvements are still required to reach the ultimate goal of directly detecting hot giant extrasolar planets.Comment: 11 pages, 6 figures, accepted for publication in A&

Quadproj: a Python package for projecting onto quadratic hypersurfaces

Quadratic hypersurfaces are a natural generalization of affine subspaces, and projections are elementary blocks of algorithms in optimization and machine learning. It is therefore intriguing that no proper studies and tools have been developed to tackle this nonconvex optimization problem. The quadproj package is a user-friendly and documented software that is dedicated to project a point onto a non-cylindrical central quadratic hypersurface